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£^ . Abstract 

Here we numerically study models of excitable media, namely, net- 
works with occasionally quiet nodes and with connection weights that 
vary with activity on a short-time scale. The networks global activ- 
ity show spontaneous (i.e., even in the absence of stimuli) unstable 
dynamics, nonequilibrium phases — including one in which the global 
activity wanders irregularly among attractors — and 1// noise as the 
system falls into the most irregular behavior. A net result is resilience 
which results in an efficient search in the model attractors space that 
can explain the origin of similar behavior in neural, genetic and ill- 
condensed matter systems. By extensive computer simulation we also 
CO ■ address a previously conjectured relation between observed power-law 

distributions and the possible occurrence of a "critical state" during 
functionality of (e.g.) cortical networks, and describe the precise na- 
ture of such criticality in the model. 
PACS: 05.10.-a, 84.35. +i, 05.45.-a, 87.19.lj, 87.18.-h, 05.45. Gg 

§ ■ 1 Introduction 

A network is said to have attractors when it can autonomously change its 
pattern of overall activity to converge with time towards one case while 
being resilient to perturbations. Following psychological observations [lj and 
formal work by an engineer [2] and a physicist [3], the concept was popular 
two decades ago as a mathematical tool to explore the fundamentals of brain 
tasks attributed to cooperation between many neurons. According to the, 
say, standard model [3], patterns of information, corresponding to sets of 
values for the nodes activity, are stored in a way that affects the intensities of 
the edges, representing synapses, which induces a heterogeneous distribution 
of the edge weights. The global activity may then converge towards one of 
the given patterns when starting with a degraded version of it. That is, the 



system exhibits kind of resilience, often known as associative memory — a 
property that mimics the process of recognizing a childhood friend we have 
not seen for dozens of years — which being common to humans is difficult to 
be efficiently emulated with computers. Such a remarkable consequence of 
cooperation is also relevant to the understanding of complexity in a variety 
of systems and to solve actual optimization problems [21 SI [SI El E]- 

The systems of interest in nature do much more than just choosing 
one out of a set of patterns and staying in its neighborhood, however (see 
[U |H1 [10] and references therein). For example, signals from the heart and 
cortical neural activities have been successfully interpreted using non-linear 
dynamic theory |ll)-|18j. and the standard model has been generalized along 
biologically-motivated lines that endow it with even more interesting behav- 
ior |19]-|26|. In particular, it was shown that one may capture some of 
the observed shaky mechanisms and instabilities by taking into account two 
features that seem to characterize generically excitable media |27j . namely, 
assuming both rapid activity-dependent fluctuations of the edge weights and 
the existence of nodes that are reluctant to a change of state during a time 
interval after operation. It is remarkable that incorporating these simple 
mechanisms into the standard model has allowed one to recreate |28) the 
transient dynamics of activity as observed in experiments concerning the 
locust odor representation |29] , 

The nervous system is definitely not the only network that exhibits both 
varying edge weights and silent nodes at a basic level of observation and, 
as a reflection of this at a higher level, roaming dynamics characterized 
by a continuous wandering among attractors. This occurs in ill-condensed 
matter, for instance, whose emerging properties are determined by "micro- 
scopic disorder". In fact, it is sensible to imagine such a disorder is more 
involved than assumed in familiar spin glass models. That is, the effec- 
tive interactions between ions should certainly be expected to have short 
time variations — associated to ion diffusion, basic chemical reactions, and 
other local changes concerning impurities, misfits, fields, rearrangements and 
strains, etc. — which would in general induce nonequilibrium patterns of ac- 
tivity as, for example, observed in reaction-diffusion systems |30l I31j . It 
is likely that the behavior of genetic networks during biological evolution 
is another case of microscopically-induced roaming dynamics |32[ [33"1 I34| . 
Furthermore, though to our knowledge the relevance of roaming has not yet 
been described for other excitable systems, it is noticeable that variability 
of connections and occasional lack of individual activity are features that 
typically characterize friendship, social, professional and business contacts 
[35] . the case of the interrelated metabolic reactions that run the cell, food 
webs, and transport and communication networked systems, for instance. 

In this paper, we describe in detail model phenomenology bearing rel- 
evance to situations with spontaneously unstable dynamics associated to 
excitability as described in the two previous paragraphs. By extensive com- 



puter simulations, we show both first and second order phase transitions, 
characterize the nature of different nonequilibrium phases [30] that occur as 
one modifies the system parameters, study the details of the network activ- 
ity dynamics, and determine the conditions in which long-range correlations 
and non-Gaussian noise emerge. This results in a systematic study that adds 
up to recent efforts trying to understand the origin of the observed relation 
between certain statistical criticality and dynamically critical functionality 
in neuroscience [9j \10\ 136"! I3T1 [38l 139] l4"0| |4"T] . Our study in this paper com- 
plements analytical study of the simplest limits of the same model in Ref . |27| 
and related exploratory numerical studies therein. 

2 Definition of model 

Consider a network in which the consequences of the activity changes of 
each node above threshold may be sketched by means of a binary variable: 
G{ = ±1, i = 1, . . . , N. This is known to suffice in practice to investigate main 
effects of cooperation in different contexts |42| . Each node receives a signal 
— alternatively, endures a local field — hi(a) = Ylj^i w ij cr j' where a ={o"i} 
stands for the global activity and W{j is the weight of the connection between 
nodes i and j. In the problems of interest, one may typically single out P 
patterns of activity, namely, {^ = ±1} with \i = 1, . . . P, that have some 
special relevance. The weights then follow accordingly, e.g., by means of the 
superposition rule Wij = i X^u=i £f £f ■ This is one of the simplest conditions 
that transforms the special P patterns into attractors of dynamics [TJ H] . 

Short-time variability of connections will be introduced by assuming that 
their weights are given by Wij = tjWij, where €j is a stochastic variable. In 
order to mimic the cases of interest, this variable should change very rapidly 
compared with the network characteristic time scale. Therefore, we shall 
assume it can be described by a stationary distribution. This is taken here 
as P s t(ej\a) = £(<r) S(ej — <&) + [1 — C(°0] $( e j ~ !)• That is, with probability 
((cr), which in general depends on the global network activity, the weights 
are changed by a factor <3? but remain unchanged otherwise. Depending 
on the value of $, this may simulate nodes excitability or potentiation or 
fatigue of the connections as a function of the degree of order in the system. 
The standard model corresponds to <i> = 1. Other choices for P st (ej\a) have 
been investigated [43], including one in which the weights change depending 
on the degree of local order, and it seems that these details do not modify 
essentially the system behavior. We shall further assume for simplicity that 
the relevant probability in P st is a sort of order parameter, namely, 

C(<T)=C(m)^ TT ^f>»] 2 . (1) 

Here, m = {to^ct), . . . ,m p (a)} is a vector whose components are the P 



overlaps of the current state a with each of the singularized patterns, namely, 

Time evolution is a consequence of transitions <7j —> ±o"j that we per- 
formed with probability i jl — ctj tanh [/ij(<t)T _1 ] } . Here, T is a parameter 
that measures the degree of stochasticity driving the evolution — the so- 
called network temperature. Another main parameter is the fraction, p, of 
nodes which is updated at each unit of time — the Monte Carlo (MC) step 
(per node). For simplicity, we shall assume here these nodes chosen at ran- 
dom from the whole set of N. In this way, the result is a situation in between 
the limits p — > (sequential or Glauber updating) and p — >■ 1 (parallel or Little 
updating). The case of intermediate p better corresponds to those situations 
in which due to excitability or other causes, e.g., power economy, not all the 
elements are active at all times. 

For simplicity, we shall be concerned only with mutually orthogonal pat- 
terns. This is achieved in practice setting every node in £? for all [i equal to 
+1 or —1 independently with the same probability, so that ^ -£f — for any 
p 7^ v in a large system. (Assuming specific sets of P correlated patterns, 
which is of great practical interest, is beyond the scope of this paper that 
intentionally understates this model detail.) Then, under some restrictions 
which strictly require also the limit p — > (see |44j for technical details), 
the conditions so far stated may be taken into account by assuming effective 
weights: 

{l-^[((m)+((m ; )]},,, (2) 

where the components of m J are m^(a) — 2a^N~ l . We shall consider in 
the following this simplified version of our model which coincides with the 
general case for any p > after averaging Wij = €jWij over the stationary 
noise distribution P st (ej\cr). As a matter of fact, ([2]) may formally be viewed 
as any learning prescription, Wij, which is affected by a multiplicative noise — 
with correlations built due to the dependence on m. Incidentally, connections 
that are roughly of this type were recently shown to induce sort of criticality 
in (neural) population dynamics [45]. 

3 Phases and diagrams 

A main observation concerns the nature of the phases exhibited as one varies 
the noise parameter, $, the fraction of active nodes, p, the temperature T, 
and the load parameter a = P/N. It turns out convenient to monitor the 
time evolution of various order parameters |46[ I47j ; in particular, 
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where the asterisk is the value of /i that identifies the pattern having the 
largest squared overlap, (m*) , and (• • • ) and (• • • ) stand, respectively, for 
averages over time and over independent realizations of the experiment (i.e., 
changing both the initial conditions and the set of the special, stored pat- 
terns). The set of the other overlaps, m^ with /i/*, may be characterized 
by: 

R= T —{ >(/»")-'). (!) 




where the sum is over all patterns excluding the one in Eq. ([3]). We also 
monitored the global activity by means of 

Our values for M, R and Q in the following involve sufficient averages of 
independent values to obtain smooth typical behavior, namely, from 200 to 
1000 MCS and 50 to 100 systems for static values, and from 10000 to 50000 
MCS and 10 systems for time-dependent values, unless indicated otherwise. 
In the standard case $ = 1, for uncorrelated patterns, the system shows 
three phases |46l 0] : 

(Phi) Memory phase, in which the system evolves towards one of the given 
patterns — often known as pure or Mattis states. The stationary state 
corresponds to maximum overlap with the particular pattern, so that 
M is large while R is small in the stationary state, namely, R ~ 
O [(P - 1)/{N + P)] . One also has that Q ~ 1 near T = 0. (This 
case is illustrated by the two top graphs in figure 1.) 

(Ph2) Mixture phase, in which a large system converges to a mixture of 
pure states, so that it exhibits some order but not associative memory. 
Therefore, one may have several relatively large overlaps, which induces 
that < M < 1 with a lower bound — due to finite size — of order of 
1/yN, while < R < (P — 1) / (1 + a) with a lower bound of order 
of (P - 1) / (N + P) . Also, Q ~ 1 near T = 0. 

(Pb.3) Disordered phase, in which the system remains completely disordered 
as dominated by thermal noise. Then, all the overlaps oscillate around 
zero, so that M ~ 0(1/VN) and R is of order (P - 1) /(N + P), and 
Q ~ in the stationary state. 

These cases correspond, respectively, to the familiar ferromagnetic, spin glass 
and paramagnetic phases that are well characterized in studies of equilibrium 
magnetic models. 

The behavior of our system is more complex than suggested by this pic- 
ture, however. A main novelty for <£ 7^ 1 is that, as illustrated in figured] the 
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Figure 1: The overlap functions m u (t) showing typical different behaviors for 
N = 1600 nodes, P = 5 patterns, noise parameter $ = —0.5, temperature 
T = 0.01 and, from top to bottom: associative memory as in Phi at (a) 
p = 0.10 (left) and (b) p = 0.30 (right); irregular roaming among patterns 
at (c) p = 0.375 (left) and (d) p = 0.40 (right) as in Pb_4; eventual jumping 
between patterns after a set of oscillations between a pattern and its negative 
(antipattern) as in Pb.5 at (e) p = 0.50 (left); and pure pattern-antipattern 
oscillations as in Pb.6 at (/) p = 0.60. 



system exhibits different types of dynamic behavior that cannot be fitted to 
the above. That is, one observes that dynamics may eventually destabilize in 
such a way that quite irregular jumping — among attractors as well as from 
one pattern to its negative (antipattern) — occurs. The observed behavior 
suggests one to define the following dynamic scenarios, say, nonequilibrium 
phases that do not occur in the standard model: 

(PI14) Irregular roaming in which the activity keeps randomly visiting the 
basins of attraction corresponding to different patterns. (This is the 
case in figures 1(c) and 1(d), i.e., the two middle graphs in figure 1.) 

(Pb.5) Irregular roaming as for Pb.4 but eventually interrupted at random 
during some time by oscillations between a pattern and its antipattern. 
(This occurs in figure 1(e).) 

(Phg) Pure pattern-antipattern oscillations. (As in figure 1(f).) 

These three genuine nonequilibrium cases correspond to Q ~ and M ~ 
(due to orthogonality). Case Phg also has R ~ (revealing the symmetry 
of oscillations), while both Pb_4 and PI15 have R 7^ 0. In order to properly 



characterize these dynamic cases, we shall monitor latter the statistics of the 
itinerant trajectory. 

The different behaviors are better observed and interpreted at very low 
temperature. As shown in figure [2j the disordered phase Pb.3 is not observed 




Figure 2: Nonequilibrium phase diagram (<£,/?) at low temperature. This 
was obtained for N = 1600, P = 5 and T = 0.1 from detailed analysis of 
all the order parameter functions. The top (blue) line is for M = 0.8. This 
leaves the equilibrium phases above, where Phi occurs with probability 0.87 
and Pb_2 otherwise. To the bottom, the next (violet) line — leaving also 
Phie above — is for M = 0.5. The next (green) lines comprise an inverted-U 
shaped region in which R > 0.18. The inset shows the roaming region in 
more detail. 



at the chosen (low) temperature, while the ordered, ferromagnetic and spin- 
glass phases then occur for any <& as far as p is not too large. That is, 
one may have familiar order as in equilibrium — practically independently 
(over a wide range) of the noise affecting the connections — as far as only a 
relatively small fraction of nodes are simultaneously active |44| . However, one 
observes small fluctuations or dispersion with time around the mean value 
M, and that the amplitude of this kind of "error" increases as one lowers <J> 
and increases p. This effect, which is evident when one compares the two 
top panels in figured! led us to indicate a zone Phie around the region for 
4> < and p < 0.5. It is worth to distinguish this zone which reveals how 
the ferromagnetic phase Phi has resilience, i.e., a remarkable stability of the 
attractor to large fluctuations. These increase monotonously with increasing 
p and/or decreasing further $, and it finally results in jumping to other 
attractors (as in the two middle graphs in figure [1]) when more than one 
half of the nodes are simultaneously active. This is the origin of the genuine 
nonequilibrium cases Pb.4, Pb.5 and Pb_6. In fact, as shown in figure [2j one 
observes the onset of irregular roaming with R 7^ and M = for <3? < 
and p between 0.4 and 0.6. 



The above picture and figure [2] follow from a detailed combined analysis 
of functions M(<&,p), R(<&,p) and Q(&,p) as illustrated in figure [3l This 
also shows that two main types of phase transitions between equilibrium 
and nonequilibrium phases occur (see figure [4]). There is a second-order or 




Figure 3: M($>,p) (left) and R(<&,p) (right; same axes but not shown for 
clarity) for N = 1600, P = 5, and T = 0.1. There is coexistence of Phi 
and Pb_2 for $ > 0, while the latter phase does not show up for $ < and 
memory then occurs but as Pb_ie (see the main text) at sufficiently low p. 



continuous transition, as one maintains $ < at a constant value, from 
the memory phase with large "error", i.e., Pb_ie, to the irregular roaming 
phase Pb.4. Then, at least near T = 0, one also observes a first-order 
or discontinuous transition (figure U]), as p is maintained constant, from the 
memory phase to the irregular roaming with pattern-antipattern oscillations, 
namely, Pb.5. Furthermore, it is noticeable here that, as illustrated in figure 
[5j the transition region depends on the value of a = P/N, that is, the critical 
value of p increases somewhat with decreasing a for finite N, and it seems 
to go to p ~ 0.5 as N — > 00 for finite a. 

The rare shape of the roaming region in plane (<J>, p) for P = 5, which 
shows in detail the inset of figure [2j is roughly the same as the one obtained 
analytically when P = 1 for the change of sign of the Lyapunov exponent 
in a closely related model (figure 2 in Ref.[27j). This confirms the general 
observation during our MC experiments of kind of chaos within the inverted- 
U region which is delimited in figure [2] by the green lines. That is, one 
should endow a chaotic character to the roaming region. That similarity 
also reinforces the reliability of our measures of order, and it shows how 
robust the model here is in relation to the dynamically irregular behavior. It 
also follows, in particular, that the model parameter P is irrelevant to this 
qualitative behavior, at least as far as not too many patterns are stored. 
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Figure 4: Left : Second-order phase transiton between Phie and Pb-4 around 
p ~ 0.37 when $ = —0.8. Right: First-order phase transition between Phi 
and Pb_5 around <£ ~ —0.1 when p = 0.8. Both plots are for N = 1600, 
P = 5, and T = 0.01. Note that different realizations using a different seed 
produce here different values corresponding to the different symbols; the 
mean of all the realizations is reperesented by a solid curve. 



The "phases" Ph4 and Pb-5, e.g., cases (d) and (e) in figure [TJ cannot be 
discriminated on the basis of M, R and Q only. The top panel in figure [6] 
illustrates how these functions change with p for fixed $ at low temperature. 
The bottom panel illustrates the dynamic transition from irregular roaming 
in Pb_4 to the more regular behavior in Pb.5 as a consequence of increasing 
the amplitude of fluctuations around the attractor as the fraction p of active 
nodes is increased during time evolution. As indicated in figure [2J the sepa- 
ration between the memory phase Phi or Phie and the nonequilibrium cases 
is clear cut, while again it results more difficult to discriminate numerically 
the region Phg of pure pattern-antipattern oscillations (where M = R = 0) 
out of the Ph4-Pb_5 chaotic region (where M = Q = with R 7^ 0). In any 
case, however, our finding concerning this agrees with the analytical result 
in a related case [2j 



4 The onset of irregularity 

The above shows that the most intriguing behavior is when the system activ- 
ity becomes irregular, e.g., as one crosses the second-order transition from 
the memory phase region to the nonequilibrium behavior — either at Pb_4 
with irregular roaming among attractors or at Phs where this may be ran- 
domly interrupted by series of pattern-antipattern oscillations. Figure [7] 




Figure 5: The second-order phase transition on the left of figure |4j For the 
same system as in this figure, the main graph here shows data for P = 5 
and N = 1600, 3200 and 6400, respectively from left to right in the middle 
of the Q value. The inset is for the same values of N but P = 5, 10 and 20, 
respectively, i.e., same value of a. 



illustrates an aspect of this transition. In addition to the time evolution of 
some of the overlaps (right panels) , which indicates where the activity is at 
each moment, this shows (left panels) the signal hi(t) that can sometimes be 
monitored in experiments. As a matter of fact, this may be compared, for 
instance, with electrical signals measured in single neurons — as well as more 
delocalized, local fields — in the cortex and hippocampus of rats [48], and 
with MEG signals and recordings for single neuron action potentials |49[ I5T)] . 
It thus seems it would be most interesting to characterize more quan- 
titatively how the model signal transforms while performing the relevant 
transitions. That is, when moving from the case of random fluctuations 
around a constant value in the memory phase, to the case in which the am- 
plitude of the fluctuations increases and eventually switches to the negative 
of the original value, and finally reaches the case in which the frequency of 
switching and all the other variables become fully irregular in Pb.4 and Pb.5. 
With this aim, we studied in detail the distribution of times of permanence 
in an interval around significative values of h. More specifically, in order to 
extract the relevant information in the case of quite different signals such 
as those in figure it turned out convenient to compute the distribution of 
time intervals, say At, in which the signal continuously stays in any of two 
ranges either h(t) > ho or h(t) < —/to- The cutoff ho intends to suppress 
the smallest fluctuations, which correspond to non-significative noise; this 
is achieved here in practice for ho £ [0.05, 0.1] . We thus observe, after aver- 
aging over the network, time and different experiments that the interesting 




Figure 6: Upper panel: Functions M(p), R(p) and Q(p) for $ = —0.7, 
T = 0.1, N = 1600 and P = 5. Bottom panel: Time series for the overlap 
functions m^i) in the same case. The value of p is increased here during time 
evolution as indicated by the horizontal axis in the upper panel. Different 
colours correspond in this graph to different values of v. 



behavior requires relatively large systems, so that it does not occur for, say, 
iV = 400 and P = 5 while it already becomes evident for, e.g., N = 6400 
and P = 40. The most interesting fact from this study is that the exponent 
f3 in a power-law fit At - ' 3 monotonously increases with size from /3 ~ 1 for 
N = 800 and P = 10 in a way that might indicate a tendency of /3 to 1.5-2 
(though our data never reached this regime). These facts are illustrated in 
the following figures. 

The left panel in figure [8] shows a changeover from a general exponential 
behavior to a power-law behavior near the interesting second-order phase 
transition. Analysis of the Fourier spectra reveals a similar situation, i.e., 
changeover from exponential to power-law behavior, concerning both the 
signal h(t) (right pannel in figure [8]) and the overlap function m{t). Figure [8] 
is a definite evidence for statistical criticality as one approaches the relevant 
transition. On the other hand, figure [9] shows how the system activity close to 
the transition between the memory equilibrium phase Phi and the irregular 
behavior in Pb-4 tends to follow the power law distribution over a larger 
range as one increases the size N for fixed P, which decreases a. However, 
we observed (not shown) that (3 does not depend on N, namely, the same 
value $ = 1.4 is obtained when P = 20 for N = 1600, 3200 and 6400. 

5 Final discussion 



Chemical reactions diffusing on a surface, forest fires with constant ignition 
of trees, parts of the nervous system vigorously reacting to weak stimuli, and 
the heart enduring tachycardia are paradigms of excitable systems — out of 
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Figure 7: The local signal or field h{t) on a typical neuron (left panels) 
and five overlaps m u (t) (right panels) indicated with different colours for a 
system with N = 1600, P = 20, $ = -0.80, T = 0.01 and, respectively from 
top to bottom, p = 0.225, 0.325 (near the transition point), and 0.425. 



many cases in mathematics, physics, chemistry and biology; see |51|, I52J . for 
instance. Despite obvious differences, these systems share some character- 
istics. They comprise spatially distributed "excitable" units connected to 
each other and cooperating to allow for the propagation of signals without 
being gradually damped by friction. The simplest realization of the relevant 
excitability consists in assuming that each element has a threshold and a re- 
fractory time between consecutive responses. In order to deal with a setting 
which is both realistic and mathematically convenient, one may suppose the 
system is networked with occasionally quiet nodes and connection weights 
that vary with activity on short-time scales. As a matter of fact, experimen- 
tal observations reveal rest states stable against small perturbations, which 
correspond to the silent nodes here, and rapid varying strength of connec- 
tions, either facilitating or impeding transmission, which temporarily affect 
thresholds and may also induce time lags during response. Furthermore, 
it is known that such nonequilibrium setting induces dynamic instabilities 
and attractors \27\ 143] , On the other hand, we believe it is likely that this 
modelling of excitable media may in fact be related to the one by means 
of partial differential equations such as when the simple FitzHugh-Nagumo 
model |53j is used to represent each unit. 

With this motivation, we have studied excitable media by extensive com- 
puter simulations of a discrete time model with an updated rule which gen- 
eralizes the Hopfield-like standard case. The resulting phenomenology as 
described here is expected to describe the basic behavior in a number of ap- 
parently diverse man-made and natural excitable systems. In particular, we 
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Figure 8: Logarithmic plots. Left: Distribution of time intervals in which 
the signal continuously stays in any of the two ranges either h (t) > ho or 
h(t) < -h , with ho = 0.1, when N = 1600, P = 20, $ = -0.8 and 
T = 0.01, for the sub-critical cases p = 0.225 (a) — a practically horizontal 
signal in the Phi phase — and 0.3 (b), the super-critical cases p = 0.35 
(d) and 0.425 (e) — an exponential behavior in the Pb_4 phase — , and the 
near-critical case p = 0.325 (c). The latter, near-critical case approximately 
follows the dotted line At - ' 3 with f} = 1.4 for a large time interval. Each 
case corresponds to an average over 50 neurons and 20 independent systems 
running for 10 5 MCS. Right: Power spectra of h(t) for the same cases as 
in the left pannel using runs with 4xl0 5 MCS. The power-law is illustrated 
with a dotted line. 



explicitly show how the model exhibits in the absence of stimuli highly un- 
stable dynamics when a sufficiently large fraction p of nodes are synchronized 
and for certain values of a noise parameter $ that controls the noise within 
the connections strength. We also illustrate how these instabilities induce the 
occurrence of novel, first- and second-order nonequilibrium phases. One of 
these happens to be most interesting as it describes the global activity wan- 
dering irregularly among a number of attractors, details strongly depending 
on the values of p and <3?. In particular, one may tune an efficient search in 
the model attractors space which is sensible to assume it may be at the origin 
of phenomenology previously described for neural, genetic and ill-condensed 
matter systems. There is also definite evidence of non-Gaussian, 1// noise 
when the system is tuned into this irregular behavior, which may explain 
recent experimental observations of criticality and power-law distributions 
in cortical networks. 

Finally, we remark how the mechanism behind the irregular jumping 
from one pattern to the other is well understood in the model. That is, the 
relevant instabilities are to be directly associated to the effective local fields 




Figure 9: The same as in ngJH] to show the effect of varying the size N at 
fixed a = P/N = 0.003125 and p = 0.375. From bottom to top, the data — 
corresponding to an average over 50 neurons and 10 independent systems — 
are for N = 1600 and 3.5 xlO 6 MCS (red), N = 3200 and 6xl0 5 MCS 
(green), and N = 6400 and 8xl0 4 MCS (blue), respectively. (For clarity 
purposes, there is a vertical translation of the data points, and it was set 
ho = 0.05 here.) Both the exponent (3 in At"' 3 as well as the cutoff at which 
this power law fails clearly increase as N is increased. 



that one may write as 



hf. 



[l-fl-^m)]^^ (6) 

for large N, i.e., neglecting terms of order N _1 . After some manipulation, 
one may write this more explicitly as 



H 



eff 



h 



Hebb 









Vfmn 



(7) 



Here, ^ ebb stands for the energy per neuron in the standard model, rj = (1 — 
<3?)/(l + a), and the last sum is over all pairs of different indexes jjl and v. As 
discussed above, h^ tends to drive the system activity near the attractor 
associated to one of the stored patterns. Together with the second term in 
Eq. (J7|), this sums up to ]T\ £^m^[l — rj(m^) 2 ] which, depending on the value 
of r], induces instabilities and irregular behavior of the overlaps dynamics 
similar to those in a cubic map |54j . The third term in ([7]), on the other hand, 
may be written as —r]^2 u m u h^ with h^ = X^/t=^ mAt £f m ^- Given that v 
differs from \x here, this only includes asymmetric terms ^m v similar to those 
that characterize the local fields for asymmetric learning rules, namely, hi = 
Yla^ t i rn ^ +l i wm ch are often used to stored and retrieve ordered sequences 
of patterns [551 HZ]- It is sensible to assume, therefore, that this term is 



most efficient in the present case in inducing transitions among patterns. 
Unlike for asymmetric learning |55j, however, the destabilization here does 
not induce any order nor predictability in the sequence of visited patterns. 
This work was supported by the Junta de Andalucia project FQM-01505, 
and by the Spanish MICINN-FEDER project FIS2009-08451. 
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